Vibration-based manufacturing plant control

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, that optimize operation of manufacturing plants by adjusting the operation of manufacturing devices in manufacturing plants based on an assessment of their operations. Methods may include obtaining, from a first set of sensors, vibration data specifying vibration in a manufacturing device of a manufacturing plant. The vibration data may be processed to identify a vibration signature. Based on the vibration signature and known vibration signatures, a first operational state of the manufacturing device may be determined. One or more operational characteristics of the manufacturing device may be adjusted based on the first operational state of the manufacturing device, to achieve a second operational state.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. § 119(e) of U.S.Patent Application No. 62/896,938, entitled “VIBRATION-BASEDMANUFACTURING PLANT CONTROL,” filed Sep. 6, 2019. The disclosure of theforegoing application is incorporated herein by reference in itsentirety for all purposes.

BACKGROUND

This specification relates to optimizing operation of manufacturingplants by adjusting the operation of manufacturing devices inmanufacturing plants based on an assessment of their conditions and/oroperations.

Industrial manufacturing plants (e.g., paper processing plants, metalprocessing plants) utilize several different types of manufacturingdevices (e.g., refiners, pellet mills, centrifuges, drills, etc.) toperform their manufacturing/production operations. A manufacturingdevice generally includes electrical, mechanical, and/or computingcomponents that can degrade with use over time. If a manufacturingdevice is not adequately monitored (e.g., for signs of wear and tear)and regularly serviced and/or repaired, the operational efficiency(e.g., the processing or manufacturing output) of the manufacturingdevice can reduce over time. In some instances, without adequatemonitoring or timely maintenance/repairs, the components of amanufacturing device can unexpectedly fail, which can result in anunplanned halting in the operation of the manufacturing device. This inturn can reduce the manufacturing output of the manufacturing plant orin some cases where the failed manufacturing device is critical to theplant's operations, completely cut off the manufacturing output of themanufacturing plant until the failed manufacturing device isrepaired/replaced.

SUMMARY

In general, one innovative aspect of the subject matter described inthis specification can be embodied in methods that include theoperations of obtaining, from a first set of sensors, vibration dataspecifying vibration in a manufacturing device of a manufacturing plant;processing the vibration data to identify a vibration signature;determining, based on the vibration signature and known vibrationsignatures, a first operational state of the manufacturing device; andadjusting, based on the first operational state of the manufacturingdevice, one or more operational characteristics of the manufacturingdevice to achieve a second operational state. Other embodiments of thisaspect include corresponding systems, apparatus, and computer programs,configured to perform the actions of the methods, encoded on computerstorage devices. These and other embodiments can each optionally includeone or more of the following features.

In some implementations, the methods can include obtaining, from asecond set of sensors, sound data representing sound present in themanufacturing plant; processing the sound data to identify a soundsignature; and determining, based on the sound signature and thevibration signature, a reinforced operational state of the manufacturingdevice, wherein adjusting one or more operational characteristics of themanufacturing device comprises adjusting the one or more operationalcharacteristics based on the reinforced operational state of themanufacturing device.

In some implementations, the sound data can represent sound emanatingfrom the manufacturing device for which the vibration data was collectedor a different manufacturing device in the manufacturing plant.

In some implementations, determining the first operational state of themanufacturing device can include determining that the operational stateis one of a replace device state, normal operational state, abnormaloperational state, predicted failure state, or optimization opportunitystate.

In some implementations, determining, based on the vibration signatureand known vibration signatures, a first operational state of themanufacturing device, can include inputting the vibration signature intoa device state model, wherein the device state model is trained on (i)historical vibration data and (ii) corresponding operational states ofmanufacturing devices, and outputs determined operational states ofmanufacturing devices based on input vibration signatures; andobtaining, from the device state model and based on the input vibrationsignature, output specifying the first operational state of themanufacturing device.

In some implementations, determining, based on the sound signature andthe vibration signature, a reinforced operational state of themanufacturing device, can include inputting the sound signature and thevibration signature into a device state model, wherein the device statemodel is trained on (i) historical vibration and sound data and (ii)corresponding operational states of manufacturing devices, and outputsoperational states of manufacturing devices based on input vibration andsounds signatures; and obtaining, from the device state model and basedon the input vibration and sound signatures, output specifying thesecond operational state of the manufacturing device.

In some implementations, the methods can include the operations ofgenerating a visualization indicating expected operational performanceof the manufacturing device based on the first operational state, thesecond operational state, or both.

In some implementations, obtaining the vibration data can includecollecting vibration data over a specified period of time and processingthe vibration data to identify a vibration signature can includedetermining the vibration signature based on a pattern of vibrationattributes over the specified period of time.

In some implementations, determining a first operational state of themanufacturing device can include inputting the vibration signature intoa device state model; and obtaining output from the device state modelindicating that the device is determined to have the first operationalstate.

Particular embodiments of the subject matter described in thisspecification can be implemented to realize one or more of the followingadvantages. The innovations described in this specification provideassessment of the current conditions/operations of manufacturingdevices. Conventionally, plant operators periodically perform visualinspections of manufacturing devices and/or semi-automated diagnostictests (which may require taking the manufacturing device offline). Theseinspections/tests are limited in that they provide fewer snapshots intime of the device's condition/operation and even then, only providelimited information about the device's condition/operation (e.g., owingto the fact that inspecting certain components of a manufacturing devicemay be difficult due to their inaccessibility) and may affect theproductivity of the plant (e.g., as a result of shutting down themanufacturing device). As a result, conventional monitoring efforts formanufacturing devices consume a significant amount of time and resourcesand generally do not provide a timely or an accurate enough assessmentof the current operational state of the manufacturing device (or even anoverview of the device's operational state over time).

In contrast, this specification provides real-time (or predictive) andaccurate assessments of the operational state of the manufacturingdevice based on real-time sensor data (e.g., data from sound and/orvibration sensors distributed around the manufacturing plant) andmodel-based analysis (e.g., models that are trained on historic sensordata and their corresponding operational states) of the sensor data.This not only reduces the amount of time and resources required toperform the monitoring, it also provides a real-time and accurateassessment of the condition/operation of the manufacturing device.Moreover, such real-time monitoring does not require shutting down orotherwise removing the manufacturing device from service, which ensuresthat the plant's output is not detrimentally affected by suchmonitoring.

In fact, the monitoring and analysis discussed herein can result in aself-healing manufacturing system that can detect and/or predictproblems associated with one or more devices that are part of themanufacturing system, and initiate change that can correct the problems,or mitigate the problems until the problems can be more fully addressed.For example, this specification describes techniques that enable anautomated adjustment of operational characteristics of the manufacturingdevice based on the current operational state of the device (which caninclude the actual or predicted current operational state of thedevice). As such, the innovations described in this specification enablefine tuning operational characteristics of the manufacturing device fora particular operational state of the device (as opposed to adjustingoperational characteristics at particular time intervals, which may notalign with the adjustment/s required for the current operational stateof the device). Relatedly, the techniques described in thisspecification enable automated adjustments of the device's operationalcharacteristics in a manner that mitigates any expected capacity oroutput reductions resulting from the current operational state (orpredicted operational state) of the manufacturing device. For example,if a motor is expected to fail within one month at the currentoperational state of the manufacturing device, the techniques describedin this specification can, e.g., reduce the operational speed of themotor by 25% to prolong the operation of the motor with limited impact(e.g., 5-10%) on the device's manufacturing/production/processingoutput, or make other appropriate changes. Moreover, the automatedadjustments of operational characteristics of manufacturing devices alsoreduce the resources (e.g., personnel and computing resources), asconventionally required, to translate/convert conventionalmonitoring/inspection results into tangible operational characteristicsadjustments that need to be implemented.

Moreover, the innovations described in this specification can utilizeultra microphones, which enable identification of operational states(e.g., defects, failures) that may not otherwise be detectable based ona human operator's supervision. Specifically, ultra microphonesgenerally have a frequency range (e.g., 100 kHZ) that is much higherthan the audible range for a human (or even regular microphones). As aresult, the ultra microphones can detect sounds that humans (or evenregular microphones) cannot detect, which in turn can be used todetermine operational states of manufacturing devices (e.g., defects,failures, etc.), which may otherwise go undetected by human operators.

In addition, the innovations described in this specification can improvecomputation efficiency of the overall system by offloading sensor dataprocessing to a plant management system—which is generally separate fromthe sensors and/or gateways deployed in the manufacturing plant. Thisenables utilizing low cost vibration and/or sound sensors that havelower processing requirements and only require small batteries, ascompared with more expensive vibration and sound sensors that includemore complex circuitry/firmware and larger batteries (which aregenerally needed to perform most of the sensor data processing at thesensor itself).

The details of one or more embodiments of the subject matter describedin this specification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example environment in which operationsof manufacturing devices in a manufacturing plant are monitored andadjusted.

FIG. 2 is a block diagram illustrating the components of FIG. 1 thatmonitor the operational state of a manufacturing device and adjust theoperational characteristics of the device based on its operationalstate.

FIG. 3 is a flow chart of an example process for monitoring theoperational state of a manufacturing device and adjusting theoperational characteristics of this device based on its operationalstate.

FIG. 4 is an example user interface visualization that is generated forpresentation on an operator device.

FIG. 5 is a block diagram of a computing system that can be used inconnection with methods described in this specification.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

This document generally relates to techniques for optimizing operationof manufacturing plants and in particular, describes a plant managementsystem that adjusts the operational characteristics of manufacturingdevices in manufacturing plants based on an assessment of real-time datarepresenting the devices' conditions/operations, historical data, and/orpredicted data.

As further described below and in more detail throughout thisspecification, the plant management system collects and processesinformation from different types of sensors, such as vibration sensors(e.g., accelerometer, proximity sensors, etc.) and/or sound sensors(e.g., high frequency microphones, such as ultra microphones)distributed at different locations in the manufacturing plant. The plantmanagement system processes the sound data to generate a sound signatureand the vibration data to generate a vibration signature. Thesesignatures are analyzed using, for example, a model (e.g., a supervisedor unsupervised machine learning model, a rules-based engine) thatdetermines the operational states of the manufacturing devices in theplant based these signatures. As described in this specification, amanufacturing device's operational state indicates a state or health ofthe manufacturing device's operation. Example operational states includea replace device state, a normal operational state, an abnormaloperational state, a predicted failure state, a maintenance state, or anoptimization opportunity state, which are further described below. Otherstates can also be defined as appropriate and/or desired.

Based on the determined operational state of the manufacturing device,the plant management system adjusts one or more operationalcharacteristics of the manufacturing device to achieve a differentoperational state (e.g., adjust operational characteristics to changefrom a maintenance required state to a normal state). As used in thisdocument, an “operational characteristic” refers to characteristic of amanufacturing device that, when adjusted, affects the operation of themanufacturing device (or some component thereof) and can include, forexample, speed, frequency, power input or output, etc. For example, thetechniques described in this specification can be used to adjust thespeed of a defective or deteriorating motor in a manufacturing device,to prolong its operation, e.g., at least until a replacement motor canbe acquired and installed.

In some instances, the plant management system can also generate avisualization (e.g., graphical representation), which can be accessed byplant operators at operator devices (e.g., computing devices). Thevisualization provides, for example, text and/or graphs (or othergraphical information) describing and/or showing the current operationalstate of the manufacturing device, the expected operational state of themanufacturing device over time, and the adjustment actions and/oroperational characteristics that the plant management system isrecommending for adjustment.

These and other features are more fully described in the descriptionsbelow.

FIG. 1 is a block diagram of an example environment 100 in which a plantmanagement system 110 monitors and adjusts the operations ofmanufacturing devices in a manufacturing plant.

The example environment 100 includes a network 120, such as a local areanetwork (LAN), a wide area network (WAN), the Internet, a mobilenetwork, or a combination thereof. The network 150 connects the plantmanagement system 110, one or more operator devices 130, and one or moremanufacturing plants 140. Although the below description describes asingle manufacturing plant 140 being controlled/monitored by the plantmanagement system 110, in some implementations, multiple manufacturingplants 140 may be controlled/monitored by the plant management system110.

Manufacturing plant 140 (which are also referred to as “plant” in thisspecification) is an industrial site at which goods, products, or othermaterials are manufactured, produced, or otherwise processed. Ingeneral, a manufacturing plant includes multiple manufacturing devices(which are also referred to in this specification as machines) that areused in the manufacturing/production process. Although the manufacturingplant 140 illustrated in FIG. 1 includes two manufacturing devices 142-Aand 142-B, manufacturing plants can include any number and type ofmanufacturing devices that are required to perform the appropriatemanufacturing/production processes of the plant 140.

Each of the manufacturing devices 142-A and 142-B has a vibration sensor144 located on it or near it. Vibration sensors 142-A and 142-B caninclude accelerometers or proximity sensors. In some implementations,additional vibration sensors can be placed on each of the manufacturingdevices. For example, vibration sensors may be placed next to or nearbearings, shafts, motors, etc.

Manufacturing plant 140 also includes a sound sensor 146. Sound sensor146 can include any sensor that can record sounds, such as a microphone.The sound sensor 146 is disposed at a location in the manufacturingplant 140 from where it can record sounds in the plant. In someimplementations, additional sound sensors 146 can be deployed around theplant, some of which are in close proximity to a set of manufacturingdevices and some that are located on one or more manufacturing devices.

The example environment 100 also includes one or more operator devices130. An operator device 130 is an electronic device that an operator ofthe plant 140 uses to, e.g., access operations of the plant, generatereports based on data (e.g., vibration and sound data for a particularmanufacturing device over a particular period) stored in the plantmanagement system 110 (as described below), and to initiate or scheduleactions that result in effecting control over certain aspects of theplant 140 (e.g., adjusting certain operational characteristics of themanufacturing devices in the plant 140). Example operator devices 130include personal computers, tablet devices, mobile communicationdevices, digital assistant devices, augmented reality devices, and otherdevices that can send and receive data over the network 104. An operatordevice 130 typically includes a user application, such as a web browser,to facilitate the sending and receiving of data over the network 120,but native applications executed by the user device 130 can alsofacilitate the sending and receiving of content over the network 120.

The example environment 100 also includes a plant management system 110,which generally monitors and controls the operations of the differentmanufacturing devices 142-A, 142-B of the plant 140. The plantmanagement system 110 obtains data from the different sensors, e.g.,vibration sensors 144 and sound sensor 146—located in the manufacturingplant 140. Data from the sensors can be transmitted via a communicationinterface (such as Bluetooth or other nearfield communication interface)to a gateway (e.g., a wireless gateway) within the plant 140, which inturn transmits this data to the plant management system 110.

As further described with reference to FIG. 2 , upon receiving thisdata, the plant management system 110 processes this data to determinethe current operational state of the manufacturing devices (e.g., 142-Aand 142-B) in the plant 140. The plant management system 110 alsodetermines the appropriate operational characteristics of manufacturingdevices 142-A and 142-B in the plant 140 based on the currentoperational state of the plant.

The plant management system 110 includes one or more front-end servers112 and one or more back-end servers 114. The front-end servers 112 cantransmit data to, and receive data from, operator devices 130 over thenetwork 120. For example, the front-end servers 112 can provide, to anoperator application 132 executing on an operator device 130, interfacesor data for presentation with the interfaces. The front-end servers 112can also receive data specifying operator's interactions with theinterfaces of the application 130. The front-end servers 112 can updatethe interfaces, provide new interfaces, and/or update the data presentedby the interfaces based on user interactions with the application 132.

The front-end servers 112 can also communicate with the back-end servers114. For example, the front-end servers 112 can identify data that is tobe processed by the back-end servers 114, e.g., data specifyingoperational characteristics to be updated in manufacturing devices, andprovide the identified data to the back-end servers 114. The front-endservers 112 can also receive data from the back-end servers 114, e.g.,data regarding current operational states of different manufacturingdevices or summary of vibration and sound data for a manufacturingdevice, and transmit the data to the operator device 130 over thenetwork 120.

The back-end servers 114 include a vibration processing apparatus 116, asound processing apparatus 118, and a modeling apparatus 120. Theoperations of these engines is summarized below and are described indetail with reference to FIGS. 2 and 3 . The vibration processingapparatus 116 processes the vibration data received from the vibrationssensors 144 and identifies vibration signatures. The sound processingapparatus 116 processes the sound data received from the sound sensor146 and identifies sound signatures. The modeling apparatus 118 acceptsthe vibration and sound signatures for a manufacturing device as inputsto a device state model that output the current operational state of thedevice. In some implementations, the modeling apparatus 118 alsoincludes a device adjustment model that accepts the device's currentoperational state to determine an adjustment action, includingoptimizations (e.g., lubrication), maintenance (e.g., filter change,fluid replacement, etc.), or other corrective action (e.g., replacementpart needed).

The sound data storage device 122 stores, for each manufacturing device,the sound data received from the sound sensor 146, the sound signaturesassociated with the device, the corresponding operational state of themanufacturing device, and any identified adjustment action. Similarly,the vibration data storage device 124 device stores, for eachmanufacturing device, the vibration data received from the vibrationsensor 144, the vibration signatures for the device, the correspondingoperational state of the manufacturing device, and any identifiedadjustment actions. In some implementations, the sound data storagedevice and the vibration storage device can be a single storage device,instead of two separate storage devices. Storage devices 122 and 124 caneach include one or more databases (or other appropriate data storagestructures) stored in one or more non-transitory data storage media(e.g., hard drive(s), flash memory, etc.).

FIG. 2 is a block diagram illustrating the components of FIG. 1 thatmonitor the operational state of manufacturing device 142-A and adjustthe operational characteristics of this device based on its operationalstate.

As shown in FIG. 2 , the vibration sensor 144 located on manufacturingdevice 142-A constantly or periodically (e.g., every second, everyminute, every five minutes, every hour, etc.) records the vibration datafor the manufacturing device 142-A and transmits (as described abovewith reference to FIG. 1 ) this vibration data 206 to the plantmanagement system 110. Similarly, the sound sensor constantly orperiodically (e.g., on the same periodic basis as vibration sensor), 146records the sound in the plant 140 (or particular parts of the plant)and transmits this sound data 208 to the plant management system 110.Although sound data 208 and vibration data 206 are shown as beingtransmitted separately, this data can be transmitted to the plantmanagement system 110 in the same transmission.

If the vibration data 206 and the sound data 208 are transmittedtogether, the plant management system 110, upon receiving this data,splits the jointly-transmitted data into vibration data 206 and thesound data 208. In some implementations, the plant management system 110splits the data into vibration data 206 and sound data 208 based on,e.g., metadata included with the jointly transmitted data thatidentifies each dataset (e.g., sound data may be identified with a<sound> tag and a vibration data may be identified with a <vibration>tag).

In some implementations, the transmitted sound and vibration data(either as a joint transmission or as separate transmission) alsoidentifies the sensor with which each respective data is associated. Forexample, the vibration data can include metadata in the form of a tagthat uniquely identifies the sensor (e.g., “Sensor ID=32114”) thatrecorded the data.

Each sensor in the plant 140 is associated with (i.e., assigned to) atleast one manufacturing device. For example, a vibration sensor isgenerally located on or near the manufacturing device is assigned tothis device. Sound sensors can be assigned to a single manufacturingdevice, a subset of manufacturing devices in the plant 140, or all themanufacturing devices of the plant. This is because a sound signaturecan represent the sound emanating from the manufacturing device (e.g.,in the case where a microphone is placed on the manufacturing device),the sound recorded from the vicinity of the manufacturing device (e.g.,in the case where a microphone is located next to one or more machines),or the sound recorded by a sensor in the plant 140, which may not be inthe vicinity of the manufacturing device (e.g., in the case where amicrophone is located near a circuit breaker or fuse box, which may beat a distant location from the manufacturing device). When the sound isrecorded by a sensor that is located on a manufacturing device, thesound sensor is assigned to that manufacturing device. When the sound isrecorded from the vicinity of one or more manufacturing device, thesound sensor may be assigned to each of these devices. When the sound isrecorded by a sensor that is distant from any of the manufacturingdevice, the sensor can be assigned to each of the manufacturing devicesin the plant 140.

An operator of the plant 140 can configure the plant management system110 to store, e.g., in a database, the associations between differentsensors and the manufacturing devices (as described above). As a result,when the sensor data is received from the plant 140 (e.g., as a jointtransmission including sound and vibration data, or as separatetransmissions of sound data 208 and vibration data 206), the plantmanagement system 140 uses the received sensor identifier to look up theassociated manufacturing device/s corresponding to the sensor data.

Once the manufacturing device/s associated with (i.e., assigned to) thereceived sensor data is identified, the plant management system 110routes the vibration data 206 to the vibration processing apparatus 116and the sound data 208 to the sound processing apparatus 118.

The vibration processing apparatus 116 processes the vibration data toidentify a vibration signature 210. A vibration signature (which mayalso be referred to as a vibration band) can be a signature derived froman assessment of the recorded vibrations at a particular snapshot intime or based on assessments of recorded vibrations over certain timeintervals (e.g., ten seconds, one minute, etc.). For example, theprocessing includes monitoring and aggregating particular vibrationfrequencies recorded by the different vibration sensors on themanufacturing device over a certain time period. As another example, theprocessing includes performing a fast Fourier Transform (FFT) of thevibration data, which generates a vibration spectrum (e.g., a plotcurve) over a suitable (N) number of discrete points that determine,among other characteristics, the velocity, acceleration, RMS, etc.

In some implementations, the vibration signature can be a mathematicalmodel representing the vibration that was captured or recorded.Alternatively, or additionally, the vibration signature can be a sparserepresentation of the vibrations detected. The sparse representation caninclude, for example, a combination of values that respectivelyrepresent frequency information, phase information, amplitudeinformation, or other information (e.g., velocity, acceleration, rootmean square (RMS)) about the characteristics of the detected vibration.In some implementations, the vibration signature can be the digitizedreal-time vibration data captured over a certain period.

Similarly, the sound processing apparatus 118 processes the sound datato identify a sound signature 212 (which may also be referred to as asound band), which too can be derived from an assessment of the recordedsounds at a particular snapshot in time or based on assessments ofrecorded sounds over certain time intervals (e.g., ten seconds, oneminute, etc.). The sound data processing can include a similartime-frequency analysis or an FFT analysis, as described above withreference to the vibration frequency analysis. As another example, thesound data can be processed to generate sound maps that identifylocations where certain sounds are concentrated. Analyzing such soundmaps can, e.g., be used to identify shifts or changes in soundlocations, which can suggest, e.g., tears or ruptures in components of amanufacturing device.

In some implementations, the sound signature can be a mathematical modelrepresenting the sound that was captured or recorded. Alternatively, oradditionally, the sound signature can be a sparse representation of thesounds detected. The sparse representation can include, for example, acombination of values that respectively represent frequency information,phase information, amplitude information, or other information (e.g.,velocity, acceleration, root mean square (RMS)) about thecharacteristics of the detected sound. In some implementations, thesound signature can be the digitized real-time sound data recorded overa period of time.

The vibration signature 210 and/or the sound signature 212 are thenprocessed by the device state model 216 of the modeling apparatus 120 todetermine an operational state of the manufacturing device 142-A. Asused in this specification, an operational state of the manufacturingdevice is an indicator of the state or health of the device's operation.Example operational states include replace device state, normaloperational state, abnormal operational state, corrective actionrequired failure state, maintenance state, and optimization opportunitystate. Each of these states is briefly described below. (1) The normaloperational state indicates that the device is healthy and operational,and that no service, maintenance, or corrective action is necessary. (2)The replace device state indicates that a particular manufacturingdevice (or a component of such device) has failed or is expected to failwithin a certain period (e.g., five days, three months, etc.). Thisstate can also identify the reason/cause of the failure or anticipatedfailure. (3) The abnormal device state indicates that the device statemodel 216 did not identify with sufficient certainty (e.g., with aprobability meeting or exceeding a certain threshold, such as 80%) anyone operational state. (4) The maintenance required state indicates thatthe manufacturing device has a maintenance issue that requirescorrective action. This state also indicates the maintenance issue andthe predicted performance or operation of the manufacturing device untila failure event. (5) The optimization opportunity state indicates thatthe manufacturing device has normal operation, but performance oroperation of the device could be optimized by taking some action (e.g.,increasing/decreasing power to the unit, lubricating, adjusting inputto/output from the unit. In some implementations, the plant managementsystem 110 can be configured to have fewer operational states oradditional operational states.

For each manufacturing device, the vibration processing apparatus 116sends the vibration signature 210 for that device to the device statemodel 216 of the modeling apparatus 120. The device state model 216accepts a vibration signature as an input, based on which it outputs theoperational state of the manufacturing device associated with thevibration signature. The device state model 216 can be implemented as alookup table, a machine learning model (e.g., supervised or unsupervisedmachine learning model) or another appropriate statistical model (e.g.,a rules-based model) that is trained on historical vibration signaturesof manufacturing devices and the corresponding operational states ofthese devices (e.g., stored in storage devices 122 and 124). As shown inFIG. 2 , based on the input vibration signature 210, which is associatedwith manufacturing device 142-A, the device state model 216 outputs thedetermined operational state of this device.

In some implementations where the device statement model 216 isimplemented as a lookup table, the lookup table stores vibrationsignatures of manufacturing devices and the corresponding operationalstates of these devices. When a vibration signature of a particularmanufacturing device is obtained, the vibration signature and theparticular manufacturing device are compared with the vibrationsignatures and the manufacturing devices stored in the lookup table. Ifthe comparison yields a match (e.g., a matching vibration signature forthe same device or the type of device), the corresponding operationalstate value stored in the lookup table is retrieved and identified asthe current operational state of the particular manufacturing device.Additionally, or alternatively, the lookup table can also store soundsignatures for manufacturing devices along with the vibrationsignatures. In such implementations, the sound and vibration signaturesfor a particular manufacturing device are input to the lookup table toobtain the corresponding operational state for that manufacturingdevice. Alternatively, one lookup table may store sound signatures formanufacturing devices in association with the device operational statesand a separate lookup table may store vibration signatures formanufacturing devices in association with the device operational states.To the extent that the operational state output by these separate lookuptables varies, such variance can be resolved or reconciled in the mannerdescribed below.

In some implementations where the device state model 216 is implementedas a machine learning model, the machine learning model can be trainedon a set of vibration signatures and the corresponding labels regardingthe current operational state of the machine. For example, the vibrationsignatures can be labeled with information about the current operationalstate of a particular manufacturing device, and subsequently labeledwith additional state information for the particular manufacturingdevice. This enables the device state model 216 to associate eachparticular vibration signature with the current operational state of theparticular manufacturing device as well as future operational states ofthe manufacturing device. In some implementations, the device statemodel 216 can accept a vibration signature as well as a rate of changein the vibration signature over a certain period to further improve theprediction performance of the model. For example, the device state model216 can be trained on training data, where each training record of thetraining data includes a vibration signature, a change (and/or rate ofchange) in the vibration signature over time, and the correspondinglabels indicating the current operational state of a particularmanufacturing device. As a result of such training, the device statemodel 216 can more accurately determine the current operational state ofthe manufacturing device (as compared with a model that is only trainedwith vibration signatures, as described above). In some implementations,the device state model 216 uses vibration signatures acquired over acertain period (e.g., using vibration signature data that can includetimestamps) to predict a future state of the particular manufacturingdevice, and in particular, output a prediction as to when (e.g., how farin the future) the particular manufacturing device will arrive at thepredicted future operational state.

In some implementations, instead of just using vibration signatures (andin some instances, the change or rate of change in the vibrationsignatures), the device state model 216 uses both sound and vibrationsignatures (and in some instances, the changes or rates of changes insuch signatures) as inputs, based on which it outputs the operationalstate of the manufacturing device. In such implementations, the devicestate model 216 is trained on historical vibration and sound signatures(and in some instances, the changes and/or rates of change in thesignatures) associated with manufacturing devices and the correspondingoperational states of these devices (e.g., stored in storage devices 122and 124). For example, for manufacturing device 142-A, the plantmanagement system 110 inputs the sound signature 212 (received from thesound processing apparatus 118) and the vibration signature 210(received from the vibration processing apparatus 116), to the devicestate model 216, which in turn outputs the operational state for thismanufacturing device. In such implementations, the device state model216 can be trained to output the current operational state of theparticular manufacturing device, the future operational state(s) of themanufacturing device, as well as a prediction as to when (e.g., how farin the future) the particular manufacturing device will arrive at thepredicted future operational state (as described above).

In some implementations, the device state model 216 can include twosub-models: one that determines operational states of manufacturingdevices based on sound signatures received from the sound processingapparatus 118 and the other that determines operational states ofmanufacturing devices based on vibration signatures received from thevibration processing apparatus 116. The sound sub-model can be a machinelearning model (e.g., supervised or unsupervised machine learning model)or another appropriate statistical model (e.g., a rules-based model)that is trained on historical sound signatures associated with/assignedto manufacturing devices (and in some instances, the change or rate ofchange of in the sound signatures over time) and the correspondingoperational states of these devices (e.g., stored in storage devices 122and 124). Similarly, the vibration sub-model, which can also be amachine learning model (or another appropriate statistical model) can betrained on historical vibration signatures for manufacturing devices(and in some instances, the change or rate of change of in the soundsignatures over time) and the corresponding operational state of thesedevices (e.g., stored in storage devices 122 and 124). In suchimplementations, each sub-model is trained to output the currentoperational state of the particular manufacturing device, the futureoperational state(s) of the manufacturing device, as well as aprediction as to when (e.g., how far in the future) the particularmanufacturing device will arrive at the predicted future operationalstate (as described above).

In such implementations, each sub-model outputs the determinedoperational state of the manufacturing device based on the input soundor vibration signature. If the operational states determined by bothsub-models match (i.e., each model outputs the same operational state ofthe manufacturing device), the modeling apparatus 120 determines that areinforced (or in other words, a high confidence) operational state isobtained for the manufacturing device. If the operational statesdetermined by the sub-models does not match (i.e., each model outputs adifferent same operational state of the manufacturing device), themodeling apparatus 120 may determine that the operational state is anabnormal state (as described above).

Alternatively, the modeling apparatus 120 can resolve or reconcile thedisparity between the operational states output by the two sub-models byapplying certain rules (or analyzing certain factors) that result in theselection of one of the operational states determined by the sub-models.As one approach, the modeling apparatus 120 can resolve the disparity bygiving more weight to the operational state determined by one sub-modelover the other. As another approach, the modeling apparatus 120 canresolve the disparity by giving more weight to the operational statethat is of higher criticality. For example, the modeling apparatus 120can apply more weight to the sound model if it determines a replacedevice state (e.g., based on a power spike that resulted in a fuse beingblown) as compared to an abnormal state determined based on thevibration sub-model. As another approach, the modeling apparatus 120 canresolve the disparity by selecting the operational state that is thesame as the last n number of operational states determined for themanufacturing device.

In some implementations, in addition to accepting sound and/or vibrationsignatures as an input, the device state model may utilize additionaloperational characteristics about the manufacturing and/or the componentof the manufacturing device in determining an operational state of themanufacturing device. For example, the device state model can be trainedto consider additional operational characteristics, e.g., operationaltemperature of the machine, machine load, hours of operations, etc.,along with the vibration and/or sound signatures to generate a morerefined operational state. Thus, by using the sound and/or vibrationsignatures along with the operational characteristics of themanufacturing device (and/or its subcomponents), the device state modelcan output a more accurate operational state of the manufacturingdevice. For example, the device state model may be trained to recognizethat a manufacturing device is operating normally even when itsvibrations are higher than an operational range because the machine isoperating at a full load (as opposed to 70-80% of the load, in whichcase the vibrations are within the expected operational range).

Based on the determined operational state for a manufacturing device,the modeling apparatus 120 determines the appropriate adjustment/s ofoperational characteristics for the manufacturing device. In someimplementations, the device state model 216 outputs the determinedoperational state of the manufacturing device to the device adjustmentmodel 218. The device adjustment model 218 takes as input the determinedoperational state of the manufacturing device, based on which it outputsa device adjustment action. As used in this specification, a deviceadjustment action is a command or an instruction that specifies theadjustment of an operational characteristic of the manufacturing device.For example, a device adjustment action can be an instruction to reducethe operational speed of a motor of the manufacturing device. As anotherexample, a device adjustment action can be an instruction to adjust thepower draw of a manufacturing device. Further adjustments can include aninstruction that results in a physical manipulation of some component onthe manufacturing device (e.g., causing a motor or a robot to move aparticular lever or knob).

The types of device adjustment actions can include recommendations orinstructions to, e.g., add a certain (e.g., an optimal amount of oilbased on the device adjustment model) oil at the bearing, balance themanufacturing device, change the operation point of a manufacturingdevice (or a component thereof), change the sealing, re-align thebearing, reduce or increase rotational speed, change filters, and adjustthe gap distance for a specific bearing. Other types of deviceadjustment actions can be specified for particular device states.

In some implementations, the device adjustment model 218 can beimplemented as a lookup table that associates operational states withcorresponding device adjustment actions. In some implementations, thedevice adjustment model 218 can be implemented as a rules-based model(or another appropriate statistical model) for which a set of rulesspecify the appropriate device adjustment action for particularoperational states. In some implementations, the device adjustment model218 can also be implemented as a machine learning model (e.g., asupervised or unsupervised model) that is trained on known deviceoperational states and corresponding device adjustment actions (e.g.,stored in storage devices 122 and 124). The machine learning model maybe preferred in scenarios involving several operational states andseveral different adjustment actions, in which case, recording allpossible relationships and/or rules between operational states andadjustment actions may not be feasible.

The modeling apparatus 120 transmits the device adjustment actioncommand 214 to the plant 140. In particular, the device controller 204of the plant 140 receives, over network 120, the device adjustmentaction 214 from the modeling apparatus 120. The device controller 204converts the device adjustment action into an electrical signal that canbe processed by the manufacturing device. For example, if the deviceadjustment action requires the motor speed of a manufacturing device tobe reduced by 50%, the device controller 204 can send a signal to themanufacturing device 142-A, e.g., specifying the frequency of the motorthat corresponds to 50% of the current operational speed. Upon receivingthis signal, the manufacturing device adjusts the specified operationalcharacteristic (e.g., adjusting operational frequency or rotationalspeed of a motor) by adjusting controls (e.g., changing physicalpositions of mechanical components, e.g., knobs, levers, and/ormodifying settings for electric or digital components) of themanufacturing device.

In some implementations, the device adjustment model 218 determines anadjustment action, which results in adjusting one or more operationalcharacteristics of the manufacturing device. This in turn results in themanufacturing device achieving (or expecting to achieve) a differentoperational state. For example, an adjustment action that results inadjusting one or more operational characteristics (such as, e.g., reducemotor speed) can alter a device's operational state from a replacedevice state (e.g., where the device is expected to fail in three days)to a replace device state (e.g., where the device is expected to failsin seven days). As another example, an adjustment action that results inadjusting one or more operational characteristics (e.g., changing oilfilter for engine) can alter a device's operational state frommaintenance required to normal. As another example, an adjustment actionthat results in adjusting one or more operational characteristics (e.g.,apply lubrication, supply more cooling to device) can alter a device'soperational state from optimization opportunity state to a normal state.

In some implementations, an operator can pre-configure, e.g., as alookup table, the expected operational states to be achieved for aparticular operational state and adjustment action. In otherimplementations, a statistical model (e.g., a machine learning model)can be trained to determine expected outcomes based a particularoperational state and adjustment action. Such a model can be trained onknown pairs of operational states and adjustment actions and thecorresponding operational states resulting from such adjustments (e.g.,data stored in storage devices 122 and 124). As a result, an adjustmentaction and the current operational state for a device may be input tosuch a model to obtain the expected operational state for the device.The expected operational state for the manufacturing device can then bestored in the storage devices 122 and/or 124.

In some implementations, the device adjustment model 218 can determinethat no adjustment action is necessary. For example, if the device'soperational state is determined to be normal or abnormal, the deviceadjustment model 218 can output that no further action is necessary, inwhich case, no adjustments to operational characteristics are made. Asanother example, if the device's operational state is determined to bereplace device, depending on the cause of the issue, the deviceadjustment model 218 can determine that no operational characteristicsneed to be adjusted and instead, the operator must order a replacementdevice.

In the preceding paragraphs, the transmission of the device adjustmentaction to the plant 140 (i.e., the device controller 204) results inautomatic adjustments of the device's operational characteristics (inthe manner described above). Although certain operationalcharacteristics can automatically be adjusted, in some scenarios, aplant operator may nevertheless configure the plant management system110 to seek confirmation from the operator (e.g., by sending a requestto the operator device 130 and waiting for a response from the operator)before sending the command to the device controller 204 to automaticallyimplement the adjustments to the operational characteristics.

In some implementations, the device adjustment actions can also specifycertain actions that require operator intervention, e.g., physicallyreplace parts and/or shut down the manufacturing device during repair.In such scenarios, the plant management system 110 sends a message tothe application running on the operator device 130, indicating thenecessary operator intervention for the repair/corrective action. As aresult, the operator performs the necessary intervening steps beforesignaling to the plant management system (e.g., via a confirmation sentfrom the operator device 130 to the plant management system) to send thedevice adjustment action to the device controller 204 to implement theremaining operational characteristic adjustments required to completethe repair/corrective action.

As described above, the plant management system 110 determines thevibration and sound signatures associated with a manufacturing device,the current operational state of the manufacturing device, theadjustments to the device's operational characteristics, and theintended operational state. This information can be stored for eachmanufacturing device in the plant in a single storage device, oralternatively, in the sound data storage device 122 and the vibrationdata storage device 124.

In some implementations, the data stored in the sound data storagedevice 122 and the vibration data storage 124 can be used to generatevisualizations that are presented in applications executing on theoperator's device 130. For example, the generated visualizations canprovide text and/or graphs (or other graphical information) that use thedata stored about a device in these storage devices to provideoperational reports specifying the current operational state of thedevice, the expected operational state of the device, any requiredadjustment actions, and expected operation of the device over time (withor without implementing the adjustment action). An example of such avisualization is shown and described with reference to FIG. 4 .

In some implementations, the plant management system 110 can alsotransmit the current operational state (as determined by the devicestate model 216) of a particular manufacturing device to the devicecontroller 204. The device controller 204 in turn provides thisinformation to the particular manufacturing device, which uses thisinformation to provide an alert. The alert can include illuminatinglights on (or near) the manufacturing device (e.g., a “status” LED)and/or generating sounds (e.g., alarms) based on the determined currentoperational state. For example, a green LED can be illuminated for the“normal operational” or “optimization opportunity” states, a yellow LEDcan be illuminated for the “abnormal operation” or “maintenancerequired” states, and a red LED can be illuminated for the “replacedevice” state. As another example, different types of sounds can besounded for each operational state of the device. Alternatively, or inaddition to generating sounds and illuminating lights, the plantmanagement system 110 can generate a notification indicating the currentoperational state of one or more manufacturing devices of the plant 140and transmit this notification via an appropriate communication channel(e.g., a text message, email, or other visualization) to an appropriateapplication executed at the operator device 130.

FIG. 3 is a flow chart of an example process 300 for monitoring theoperational state of a manufacturing device and adjusting theoperational characteristics of this device based on its operationalstate. Operations of the process 300 can be implemented, for example, bythe system components shown in FIGS. 1 and 2 , and/or one or more dataprocessing apparatus. In some implementations, operation of the process300 can be implemented as instructions stored on a non-transitorycomputer readable medium, where execution of the instructions by one ormore data processing apparatus cause the one or more data processingapparatus to perform operations of the process 300.

Sensor data associated with a manufacturing device can be collected froma set of sensors (302). The set of sensors can include vibration sensors(e.g., accelerometers, proximity sensors) and sound sensors (e.g.,microphones). As described above with reference to FIGS. 1 and 2 , thesound sensors record sounds (e.g., sounds emanating from a manufacturingdevice, sounds in the manufacturing plant) and the vibration sensorrecords vibrations of the manufacturing device.

The sensor data obtained at operation 302 is processed to identifyvibration and sound signatures (304). As described above with referenceto FIGS. 1 and 2 , the sensor data—including the sound data and thevibration data—associated with the manufacturing device is transmittedto the sound processing apparatus 118 and the vibration processingapparatus 116 of the plant management system 110, which then process thereceived data to generate sound and vibration signatures, respectively.

An operational state of the manufacturing device is determined based onthe vibration and/or sound signatures (306). As described above withreference to FIGS. 1 and 2 , the vibration signature associated with themanufacturing device is input to a device state model 216, which outputsthe current operational state of the manufacturing device. In someimplementations, a separate sub-model of the device state model alsoaccepts the sound signature as an input and outputs an operational statefor the device, which is then reconciled and/or combined with theoperational state determined based on the vibration signatures (asdescribed above with reference to FIGS. 1 and 2 ). In otherimplementations, both the vibration and sound signatures are input tothe device state model 216, which then determines an operational stateof the manufacturing device (as described above with reference to FIGS.1 and 2 ) for these dual inputs.

One or more operational characteristics of the manufacturing device areadjusted based on the determined operational state (308). As describedabove with reference to FIGS. 1 and 2 , the device adjustment model 218accepts as input the determined operational state for the manufacturingdevice, based on which it generates an adjustment action command 214.The device adjustment model 218 transmits this adjustment actioncommand, via network 120, to the device controller 204 of the plant 140,which in turn generates appropriate signals to adjust one or moreoperational characteristics of the manufacturing device (as describedabove with reference to FIGS. 1 and 2 ). In some implementations, thedevice adjustment model 218 outputs an adjustment action command thatresults in changing the device's operational state from the currentstate (e.g., replace device state—failure in three months) to adifferent state (e.g., normal, replace device state—failure in sixmonths).

A visualization is generated regarding the operational performance ofthe manufacturing device (310). In some implementations, the datadetermined by the vibration/sound processing apparatuses and themodeling apparatus is stored in the sound data storage device 122 andthe vibration data storage 124. This data can then be used by thefront-end server 112 of the plant management system 110 to generatevisualizations that are presented in an application executing on theoperator's device 130. For example, the generated visualizations canprovide text and/or graphs (or other graphical information) that use thedata stored about a manufacturing device in the storage devices 122 and124 to provide operational reports specifying the current operationalstate of the device, the expected operational state of the device, anyrequired adjustment actions, and expected operation of the device overtime. An example of such a visualization is shown and described withreference to FIG. 4 .

FIG. 4 is an example user interface visualization 400 that is generatedfor presentation on an operator device.

The front end server 112 of the plant management system 110 generatesthe visualization 400 based on the data analyzed and determined by theplant management system 110 (as described above with reference to FIGS.1 and 2 ). The visualization 400 include an operational report 420 and agraph 410 showing the operational performance of the manufacturingdevice over time.

The operation report 420 identifies the manufacturing device by itsidentifier and includes the current operational state of “Replace Device(1 week),” which is determined by the device state model 216 (asdescribed above with reference to FIGS. 2 and 3 ). The operationalreport 420 also identifies the adjustment action (“Reduce operationalspeed by 50%”) and the expected operational state based on theadjustment action (“Replace Device (2 weeks)”).

The graph 410 is generated based on the historic operational statesstored for the particular manufacturing device. The graph 410 alsoextrapolates the expected operational performance until the device'sexpected failure in the future (assuming that the adjustment action isnot performed). This future extrapolation can be generated based onknown operational performance of other similar devices with similaroperation over the lifetime of the devices. The graph 410 alsoidentifies the current operational state 430 and includes additionalinformation about the cause of the issue/defect for the currentoperational state 430 (“Critical failure-causing damage to a bearing”).

FIG. 5 is a block diagram of computing devices 500, 550 that may be usedto implement the systems and methods described in this document, eitheras a client or as a server or plurality of servers. Computing device 500is intended to represent various forms of digital computers, such aslaptops, desktops, workstations, personal digital assistants, servers,blade servers, mainframes, and other appropriate computers. Computingdevice 550 is intended to represent various forms of mobile devices,such as personal digital assistants, cellular telephones, smartphones,smartwatches, head-worn devices, and other similar computing devices.The components shown here, their connections and relationships, andtheir functions, are meant to be exemplary only, and are not meant tolimit implementations described and/or claimed in this document.

Computing device 500 includes a processor 502, memory 504, a storagedevice 506, a high-speed interface 508 connecting to memory 504 andhigh-speed expansion ports 510, and a low speed interface 512 connectingto low speed bus 514 and storage device 506. Each of the components 502,504, 506, 508, 510, and 512, are interconnected using various busses,and may be mounted on a common motherboard or in other manners asappropriate. The processor 502 can process instructions for executionwithin the computing device 500, including instructions stored in thememory 504 or on the storage device 506 to display graphical informationfor a GUI on an external input/output device, such as display 516coupled to high speed interface 508. In other implementations, multipleprocessors and/or multiple buses may be used, as appropriate, along withmultiple memories and types of memory. Also, multiple computing devices500 may be connected, with each device providing portions of thenecessary operations (e.g., as a server bank, a group of blade servers,or a multi-processor system).

The memory 504 stores information within the computing device 500. Inone implementation, the memory 504 is a computer-readable medium. In oneimplementation, the memory 504 is a volatile memory unit or units. Inanother implementation, the memory 404 is a non-volatile memory unit orunits.

The storage device 506 is capable of providing mass storage for thecomputing device 400. In one implementation, the storage device 506 is acomputer-readable medium. In various different implementations, thestorage device 506 may be a hard disk device, an optical disk device, ora tape device, a flash memory or other similar solid state memorydevice, or an array of devices, including devices in a storage areanetwork or other configurations. In one implementation, a computerprogram product is tangibly embodied in an information carrier. Thecomputer program product contains instructions that, when executed,perform one or more methods, such as those described above. Theinformation carrier is a computer- or machine-readable medium, such asthe memory 504, the storage device 506, or memory on processor 502.

The high-speed controller 508 manages bandwidth-intensive operations forthe computing device 500, while the low speed controller 512 manageslower bandwidth-intensive operations. Such allocation of duties isexemplary only. In one implementation, the high-speed controller 508 iscoupled to memory 504, display 516 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 510, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 512 is coupled to storage device 506 and low-speed expansionport 514. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 500 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 520, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 524. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 522. Alternatively, components from computing device 500 may becombined with other components in a mobile device (not shown), such asdevice 550. Each of such devices may contain one or more of computingdevice 500, 550, and an entire system may be made up of multiplecomputing devices 500, 550 communicating with each other.

Computing device 550 includes a processor 552, memory 564, aninput/output device such as a display 554, a communication interface566, and a transceiver 568, among other components. The device 550 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 550, 552,564, 554, 566, and 568, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate. The processor 552 can process instructionsfor execution within the computing device 550, including instructionsstored in the memory 564. The processor may also include separate analogand digital processors. The processor may provide, for example, forcoordination of the other components of the device 550, such as controlof user interfaces, applications run by device 550, and wirelesscommunication by device 550.

Processor 552 may communicate with a user through control interface 558and display interface 556 coupled to a display 554. The display 554 maybe, for example, a TFT LCD display or an OLED display, or otherappropriate display technology. The display interface 556 may compriseappropriate circuitry for driving the display 554 to present graphicaland other information to a user. The control interface 558 may receivecommands from a user and convert them for submission to the processor552. In addition, an external interface 562 may be provided incommunication with processor 552, so as to enable near areacommunication of device 550 with other devices. External interface 562may provide, for example, for wired communication (e.g., via a dockingprocedure) or for wireless communication (e.g., via Bluetooth or othersuch technologies).

The memory 564 stores information within the computing device 550. Inone implementation, the memory 564 is a computer-readable medium. In oneimplementation, the memory 564 is a volatile memory unit or units. Inanother implementation, the memory 564 is a non-volatile memory unit orunits. Expansion memory 574 may also be provided and connected to device550 through expansion interface 572, which may include, for example, aSIMM card interface. Such expansion memory 574 may provide extra storagespace for device 550, or may also store applications or otherinformation for device 550. Specifically, expansion memory 574 mayinclude instructions to carry out or supplement the processes describedabove, and may include secure information also. Thus, for example,expansion memory 574 may be provided as a security module for device550, and may be programmed with instructions that permit secure use ofdevice 550. In addition, secure applications may be provided via theSIMM cards, along with additional information, such as placingidentifying information on the SIMM card in a non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, asdiscussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 564, expansionmemory 574, or memory on processor 552.

Device 550 may communicate wirelessly through communication interface566, which may include digital signal processing circuitry wherenecessary. Communication interface 566 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 568. In addition, short-range communication may occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, GPS receiver module 570 may provide additional wireless datato device 550, which may be used as appropriate by applications runningon device 550.

Device 550 may also communicate audibly using audio codec 560, which mayreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 560 may likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 550. Suchsound may include sound from voice telephone calls, may include recordedsound (e.g., voice messages, music files, etc.) and may also includesound generated by applications operating on device 550.

The computing device 550 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 580. It may also be implemented as part of asmartphone 582, personal digital assistant, or other similar mobiledevice.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs, computer hardware, firmware, software, and/orcombinations thereof. These various implementations can includeimplementation in one or more computer programs that are executableand/or interpretable on a programmable system including at least oneprogrammable processor, which may be special or general purpose, coupledto receive data and instructions from, and to transmit data andinstructions to, a storage system, at least one input device, and atleast one output device.

These computer programs, also known as programs, software, softwareapplications or code, include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device, e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device,e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor,for displaying information to the user and a keyboard and a pointingdevice, e.g., a mouse or a trackball, by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback, e.g., visual feedback,auditory feedback, or tactile feedback; and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component, e.g., as a dataserver, or that includes a middleware component such as an applicationserver, or that includes a front end component such as a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here, or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication such as, acommunication network. Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

As used in this specification, the term “module” is intended to include,but is not limited to, one or more computers configured to execute oneor more software programs that include program code that causes aprocessing unit(s)/device(s) of the computer to execute one or morefunctions. The term “computer” is intended to include any dataprocessing or computing devices/systems, such as a desktop computer, alaptop computer, a mainframe computer, a personal digital assistant, aserver, a handheld device, a smartphone, a tablet computer, anelectronic reader, or any other electronic device able to process data.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made without departing fromthe spirit and scope of the invention. Accordingly, other embodimentsare within the scope of the following claims. While this specificationcontains many specific implementation details, these should not beconstrued as limitations on the scope of what may be claimed, but ratheras descriptions of features that may be specific to particularembodiments. Certain features that are described in this specificationin the context of separate embodiments can also be implemented incombination in a single embodiment.

Conversely, various features that are described in the context of asingle embodiment can also be implemented in multiple embodimentsseparately or in any suitable subcombination. Moreover, althoughfeatures may be described above as acting in certain combinations andeven initially claimed as such, one or more features from a claimedcombination can in some cases be excised from the combination, and theclaimed combination may be directed to a subcombination or variation ofa subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various system modulesand components in the embodiments described above should not beunderstood as requiring such separation in all embodiments, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, some processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults.

What is claimed is:
 1. A method comprising: obtaining, by one or morecomputers and from a first set of sensors, vibration data specifyingvibration in a manufacturing device during operation of themanufacturing device in a manufacturing plant; processing, by the one ormore computers, the vibration data to identify a current vibrationsignature for the manufacturing device; determining, by the one or morecomputers and using the current vibration signature and known vibrationsignatures, a current operational state of the manufacturing device; andadjusting, by the one or more computers and based on the currentoperational state of the manufacturing device, one or more operationalcharacteristics of the manufacturing device in real time and duringoperation of the manufacturing device, comprising: in response todetermining the current operational state is a first operational stateof the manufacturing device using the current vibration signature,providing control signals to the manufacturing device for updating oneor more first operational characteristics of the manufacturing device toachieve a second operational state; and in response to determining thecurrent operational state is a third operational state of themanufacturing device using the current vibration signature, providingcontrols signals to the manufacturing device for updating one or moresecond operational characteristics of the manufacturing device toachieve a fourth operational state, wherein, the first operationalstate, the second operational state, the third operational state, andthe fourth operational state are different from each other, and whereinthe first operational characteristics are different from the secondoperational characteristics.
 2. The method of claim 1, furthercomprising: obtaining, from a second set of sensors, sound datarepresenting sound present in the manufacturing plant; processing thesound data to identify a sound signature; determining, based on thesound signature and the current vibration signature, the currentoperational state is a reinforced operational state of the manufacturingdevice; and adjusting one or more third operational characteristics ofthe manufacturing device based on the reinforced operational state ofthe manufacturing device.
 3. The method of claim 2, wherein the sounddata represents sound emanating from the manufacturing device for whichthe vibration data was collected or a different manufacturing device inthe manufacturing plant.
 4. The method of claim 1, wherein determiningthe current operational state of the manufacturing device comprisesdetermining that the current operational state is one of a replacedevice state, normal operational state, abnormal operational state,predicted failure state, or optimization opportunity state.
 5. Themethod of claim 1, wherein determining, based on the current vibrationsignature and known vibration signatures, the current operational stateof the manufacturing device, comprises: inputting the current vibrationsignature into a device state model, wherein the device state model istrained on (i) historical vibration data and (ii) correspondingoperational states of manufacturing devices, and outputs determinedoperational states of manufacturing devices based on input vibrationsignatures; and obtaining, from the device state model and based on theinput current vibration signature, output specifying the currentoperational state of the manufacturing device.
 6. The method of claim 2,wherein determining, based on the sound signature and the currentvibration signature, the reinforced operational state of themanufacturing device, comprises: inputting the sound signature and thecurrent vibration signature into a device state model, wherein thedevice state model is trained on (i) historical vibration and sound dataand (ii) corresponding operational states of manufacturing devices, andoutputs operational states of manufacturing devices based on inputvibration and sounds signatures; and obtaining, from the device statemodel and based on the input current vibration signature and soundsignature, output specifying the second operational state of themanufacturing device.
 7. The method of claim 1, further comprising:generating a visualization indicating expected operational performanceof the manufacturing device based on the (i) first operational state,(ii) the second operational state, (iii) the third operational state, or(iv) any combination thereof.
 8. The method of claim 1, wherein:obtaining the vibration data comprises collecting vibration data over aspecified period of time; processing the vibration data to identify thecurrent vibration signature comprises determining the current vibrationsignature based on a pattern of vibration attributes over the specifiedperiod of time; and determining the current operational state of themanufacturing device comprises: inputting the current vibrationsignature into a device state model; and obtaining output from thedevice state model indicating that the manufacturing device isdetermined to have the first operational state or the third operationalstate.
 9. The method of claim 1, wherein adjusting the one or more firstoperational characteristics of the manufacturing device and the one ormore second operational characteristics of the manufacturing devicecomprises: providing, to the manufacturing device, device adjustmentaction commands.
 10. The method of claim 9, wherein adjusting the one ormore first operational characteristics of the manufacturing device andthe one or more second operational characteristics of the manufacturingdevice comprises adjusting physical, electric, and/or digital controlsof the manufacturing device.
 11. A system comprising: one or more memorydevices storing instructions; and one or more data processing apparatusthat are configured to interact with the one or more memory devices, andupon execution of the instructions, perform operations including:obtaining, from a first set of sensors, vibration data specifyingvibration in a manufacturing device during operation of themanufacturing device in a manufacturing plant; processing the vibrationdata to identify a current vibration signature for the manufacturingdevice; determining, using the current vibration signature and knownvibration signatures, a current operational state of the manufacturingdevice; and adjusting, based on the current operational state of themanufacturing device, one or more operational characteristics of themanufacturing device, comprising: in response to determining the currentoperational state is a first operational state of the manufacturingdevice using the current vibration signature, providing control signalsto the manufacturing device for updating one or more first operationalcharacteristics of the manufacturing device to achieve a secondoperational state; and in response to determining the currentoperational state is a third operational state of the manufacturingdevice using the current vibration signature, providing control signalsto the manufacturing device for updating one or more second operationalcharacteristics of the manufacturing device to achieve a fourthoperational state, wherein, the first operational state, the secondoperational state, the third operational state, and the fourthoperational state are different from each other, and wherein the firstoperational characteristics are different from the second operationalcharacteristics.
 12. The system of claim 11, wherein the one or moredata processing apparatus are configured to perform operations furthercomprising: obtaining, from a second set of sensors, sound datarepresenting sound present in the manufacturing plant; processing thesound data to identify a sound signature; determining, based on thesound signature and the current vibration signature, the currentoperational state is a reinforced operational state of the manufacturingdevice; and adjusting one or more third operational characteristics ofthe manufacturing device based on the reinforced operational state ofthe manufacturing device.
 13. The system of claim 12, wherein the sounddata represents sound emanating from the manufacturing device for whichthe vibration data was collected or a different manufacturing device inthe manufacturing plant.
 14. The system of claim 11, wherein determiningthe current operational state of the manufacturing device comprisesdetermining that the current operational state is one of a replacedevice state, normal operational state, abnormal operational state,predicted failure state, or optimization opportunity state.
 15. Thesystem of claim 11, wherein determining, based on the current vibrationsignature and known vibration signatures, the current operational stateof the manufacturing device, comprises: inputting the current vibrationsignature into a device state model, wherein the device state model istrained on (i) historical vibration data and (ii) correspondingoperational states of manufacturing devices, and outputs determinedoperational states of manufacturing devices based on input vibrationsignatures; and obtaining, from the device state model and based on theinput current vibration signature, output specifying the currentoperational state of the manufacturing device.
 16. The system of claim12, wherein determining, based on the sound signature and the currentvibration signature, the reinforced operational state of themanufacturing device, comprises: inputting the sound signature and thecurrent vibration signature into a device state model, wherein thedevice state model is trained on (i) historical vibration and sound dataand (ii) corresponding operational states of manufacturing devices, andoutputs operational states of manufacturing devices based on inputvibration and sounds signatures; and obtaining, from the device statemodel and based on the input current vibration signature and soundsignature, output specifying the second operational state of themanufacturing device.
 17. The system of claim 11, wherein the one ormore data processing apparatus are configured to perform operationsfurther comprising: generating a visualization indicating expectedoperational performance of the manufacturing device based on the (i)first operational state, (ii) the second operational state, (iii) thethird operational state, or (iv) any combination thereof.
 18. The systemof claim 11, wherein: obtaining the vibration data comprises collectingvibration data over a specified period of time; processing the vibrationdata to identify the current vibration signature comprises determiningthe current vibration signature based on a pattern of vibrationattributes over the specified period of time; and determining thecurrent operational state of the manufacturing device comprises:inputting the current vibration signature into a device state model; andobtaining output from the device state model indicating that themanufacturing device is determined to have the first operational stateor the third operational state.
 19. A non-transitory computer readablemedium storing instructions that, when executed by one or more dataprocessing apparatus, cause the one or more data processing apparatus toperform operations comprising: obtaining, by one or more computers andfrom a first set of sensors, vibration data specifying vibration in amanufacturing device during operation of the manufacturing device in amanufacturing plant; processing, by the one or more computers, thevibration data to identify a current vibration signature for themanufacturing device; determining, by the one or more computers andusing the current vibration signature and known vibration signatures, acurrent operational state of the manufacturing device; and adjusting, bythe one or more computers and based on the current operational state ofthe manufacturing device, one or more operational characteristics of themanufacturing device in real time and during operation of themanufacturing device, comprising: in response to determining the currentoperational state is a first operational state of the manufacturingdevice using the current vibration signature, providing control signalsto the manufacturing device for updating one or more first operationalcharacteristics of the manufacturing device to achieve a secondoperational state; and in response to determining the currentoperational state is a third operational state of the manufacturingdevice using the current vibration signature, providing controls signalsto the manufacturing device for updating one or more second operationalcharacteristics of the manufacturing device to achieve a fourthoperational state, wherein, the first operational state, the secondoperational state, the third operational state, and the fourthoperational state are different from each other, and wherein the firstoperational characteristics are different from the second operationalcharacteristics.
 20. The non-transitory computer readable medium ofclaim 19, further comprising: obtaining, from a second set of sensors,sound data representing sound present in the manufacturing plant;processing the sound data to identify a sound signature; determining,based on the sound signature and the current vibration signature, thecurrent operational state is a reinforced operational state of themanufacturing device; and adjusting one or more third operationalcharacteristics of the manufacturing device based on the reinforcedoperational state of the manufacturing device.
 21. The non-transitorycomputer readable medium of claim 20, wherein the sound data representssound emanating from the manufacturing device for which the vibrationdata was collected or a different manufacturing device in themanufacturing plant.
 22. The non-transitory computer readable medium ofclaim 19, wherein determining the current operational state of themanufacturing device comprises determining that the current operationalstate is one of a replace device state, normal operational state,abnormal operational state, predicted failure state, or optimizationopportunity state.
 23. The non-transitory computer readable medium ofclaim 19, wherein determining, based on the current vibration signatureand known vibration signatures, the current operational state of themanufacturing device, comprises: inputting the current vibrationsignature into a device state model, wherein the device state model istrained on (i) historical vibration data and (ii) correspondingoperational states of manufacturing devices, and outputs determinedoperational states of manufacturing devices based on input vibrationsignatures; and obtaining, from the device state model and based on theinput current vibration signature, output specifying the currentoperational state of the manufacturing device.
 24. The non-transitorycomputer readable medium of claim 20, wherein determining, based on thesound signature and the current vibration signature, the reinforcedoperational state of the manufacturing device, comprises: inputting thesound signature and the current vibration signature into a device statemodel, wherein the device state model is trained on (i) historicalvibration and sound data and (ii) corresponding operational states ofmanufacturing devices, and outputs operational states of manufacturingdevices based on input vibration and sounds signatures; and obtaining,from the device state model and based on the input current vibrationsignature and sound signature, output specifying the second operationalstate of the manufacturing device.
 25. The non-transitory computerreadable medium of claim 19, wherein the one or more data processingapparatus are configured to perform operations further comprising:generating a visualization indicating expected operational performanceof the manufacturing device based on the (i) first operational state,(ii) the second operational state, (iii) the third operational state, or(iv) any combination thereof.
 26. The non-transitory computer readablemedium of claim 19, wherein: obtaining the vibration data comprisescollecting vibration data over a specified period of time; processingthe vibration data to identify the current vibration signature comprisesdetermining the current vibration signature based on a pattern ofvibration attributes over the specified period of time; and determiningthe current operational state of the manufacturing device comprises:inputting the current vibration signature into a device state model; andobtaining output from the device state model indicating that themanufacturing device is determined to have the first operational stateor the third operational state.